The present invention mainly relates to a system and a method for detecting an obstacle on a road such as a vehicle going ahead, a parking vehicle, or a pedestrian by vehicle-mounted camera system for realization of supporting safety driving of a vehicle or automatic running of a vehicle.
Technologies for detecting an obstacle may be roughly classified into that of utilizing a laser beam or an ultrasonic wave, and that of utilizing a TV camera. Utilization of a laser beam is expensive. Utilization of an ultrasonic wave causes a problem in detection accuracy of an obstacle because of a low resolution of the ultrasonic wave. In addition, with an active sensor utilizing a laser beam or an ultrasonic wave, recognition of a lane is impossible only by the sensor itself.
Compared with this, a TV camera is relatively inexpensive and also is suitable for detecting an obstacle in resolution, measuring accuracy, and a measurable range. A TV camera utilizing system may be classified into that of utilizing a single TV camera and that of utilizing a plurality of TV cameras (stereo camera).
The system utilizing a single TV camera uses a method in which a road region and an obstacle region is separated on the basis of key information about brightness, color, or textures, which are obtained from an image picked up by the camera.
For example, a region with medium brightness with low chroma, i.e. a gray region, is extracted to obtain a road region, or a region with a few textures is obtained to provide the rest as an obstacle region. There are, however, many obstacles having similar brightness, color, and textures to those of the road to make it difficult to use the method for distinguishing the obstacle region and the road region.
Compared with this, in a method used in the system utilizing a plurality of cameras an obstacle is detected on the basis of three-dimensional information as a key. The method is generally called as “stereopsis”.
The stereoscopic viewing is a method in which two cameras, for example, are disposed on the right and left, respectively, a relationship between images by the right and left cameras is obtained about the same point in three-dimensional space, and thus, a three-dimensional position of the point is obtained by adopting a procedure of triangulation. The position and an attitude of each camera about the road that are obtained beforehand allows a height from the road plane of any given point in the image to be obtained by the stereoscopic viewing. This can distinguish the obstacle region from the road region by knowing as to whether a point is positioned at some height or not. The stereoscopic viewing makes it possible to avoid a problem caused in the case where a single camera is used.
In a normal stereoscopic viewing, there is a problem in that a process of carrying out matching is necessary. The stereoscopic viewing is in general a technology of obtaining a three-dimensional position of any given point given in an image about a coordinate system fixed in the stereo camera (hereinafter referred to as a stereo camera coordinate system). The matching means a searching calculation necessitated when making a point being the same in the space into correspondence between right and left images with a problem of extremely high in calculation cost. Thus, the matching is a factor of preventing the stereoscopic viewing from being brought into realization.
In this respect, each of procedures disclosed in Japanese Patent Laid-Open No. 2001-76128 and Japanese Patent Laid-Open No. 2000-293693 provides a method of detecting an obstacle on a road (hereinafter referred to as a related procedure) with a very high speed without carrying out the matching processing even though a stereo camera is used.
In the above related procedures, a road surface is supposed to be a flat plane, an image transform T is obtained in which a picture element point (pixel) of a road surface region in one camera image (a first camera's image) is made into correspondence with a corresponding pixel of a road surface region in the other camera image (a second camera's image) on the basis of a geometrical relation between the stereo camera and the road surface, and an obstacle is detected from a difference between a transformed image by the image transform T (transformed image) and the second camera's image. That is, any given picture element on the road surface region displayed in the first camera's image is correctly transformed to a corresponding picture element in the second camera's image by the image transform T. Contrary to this, a picture element on a region of an object with any given height (i.e. an obstacle) is not correctly transformed into a corresponding picture element. Therefore, an obstacle is detected at a high speed by obtaining a difference between a position of the pixel elements in the second camera's image and that in the transformed image.
However, in the case where reflected images of an obstacle, structures around the road, and scenery of environment are produced on the road surface as observed in a wet road surface in rainy weather, it was impossible by the related procedure which was for detecting an object with a height other than zero to correctly detect an obstacle.
This is because the reflected image from the road surface can be virtually regarded as an object with a negative height, and no virtual image can be correctly transformed to a corresponding picture element by the above image transform.
Accordingly, the present invention was made in view of the above situation with an object of providing a system and a method of detecting an obstacle which has a feature of necessitating no stereoscopic correspondence like in the related procedure and is still capable of correctly detecting the obstacle even in the case where a reflected image from a road surface is produced.